Augmented reality system for vehicle blind spot prevention

ABSTRACT

The present disclosure relates to systems and methods for providing various types of information to a vehicle driver. Such information can be used by the vehicle driver singularly or in conjunction with other information available to the vehicle driver in order to allow the driver to operate the vehicle in an increasingly safe manner and/or to reduce the likelihood of property damage and/or possible bodily injuries to the driver, etc. In some instances, such information is presented to the driver as an augmented reality environment such that the driver can “see through” objects that may be occluding the driver&#39;s vision.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of application Ser. No. 14/751,891,filed Jun. 26, 2015, the entire disclosure of which is herebyincorporated by reference herein for all purposes.

BACKGROUND

People are more mobile than ever before. The number of cars, trucks,buses, recreational vehicles, and sport utility vehicles (collectively“automobiles”) on the road appears to increase with each passing day.Moreover, the ongoing transportation explosion is not limited toautomobiles. A wide variety of different vehicles such as motorcycles,trains, light, medium, and heavy duty trucks, construction equipment,and other transportation devices (collectively “vehicles”) are used tomove people and cargo from place to place. While there are manyadvantages to our increasingly mobile society, there are also costsassociated with the explosion in the number and variety of vehicles.

Accidents are one example of such a cost. The vehicle and automobileindustry is continually searching for ways to reduce accidents and/orseverity of such accidents.

SUMMARY

Embodiments of the present disclosure address the aforementioned needand others by providing various types of information to the vehicledriver. Such information can be used by the vehicle driver singularly orin conjunction with other information available to the vehicle driver inorder to allow the driver to operate the vehicle in an increasingly safemanner and/or to reduce the likelihood of property damage and/orpossible bodily injuries to the driver, etc. In some embodiments, aswill be described in more detail below, such information is presented tothe driver as an augmented reality environment such that the driver can“see through” objects that may be occluding the driver's vision.

In accordance with an aspect of the present disclosure, a methodimplemented in computer-executable instructions for displayinginformation about vehicle surroundings to the driver of the vehicle isprovided. The method includes obtaining vehicle environment data fromone or more information sources. The vehicle environment data in oneembodiment is indicative of at least a part of a scene occluded fromview of a driver when operating the vehicle. The method also includespresenting to the driver of the vehicle, with the aid of the one or moredisplays, an augmented reality environment based on the vehiclesurroundings data and representative of an area surrounding the vehiclebut obstructed from operator view.

In accordance with another aspect of the present disclosure, acomputer-readable medium is provided having modules for conveyinginformation to a vehicle driver regarding vehicle surroundings. The oneor more modules includes an information gathering module configured tocollect data from one or more information sources associated with one ormore sensing zones, an augmented reality image rendering moduleconfigured to generate from the collected data one or more virtualdesign elements representative of objects occluded from view of thevehicle driver, and a display module configured to cause the virtualdesign elements to be presented to a display.

In accordance with another aspect of the present disclosure, a system isdisclosed for providing information to a vehicle driver. The systemincludes one or more displays, one or more information sourcesconfigured to generate data associated with one or more sensing zoneslocated at one or more areas surrounding the vehicle, and a displaygenerator coupled to the one or more information sources. The displaygenerator in one embodiment is configured to: collect data from one ormore information sources associated with one or more sensing zones;generate from the collected data one or more virtual design elementsrepresentative of objects occluded from view of the vehicle driver; andcause the virtual design elements to be presented to a display.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features ofthe claimed subject matter, nor is it intended to be used as an aid indetermining the scope of the claimed subject matter.

DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of theclaimed subject matter will become more readily appreciated as the samebecome better understood by reference to the following detaileddescription, when taken in conjunction with the accompanying drawings,wherein:

FIG. 1 is a block diagram of one representative embodiment of a vehiclesafety system in accordance with one or more aspects of the presentdisclosure;

FIG. 2 is a schematic diagram illustrating a number of blind spotsexperienced by drivers of one representative vehicle;

FIG. 3 is a schematic diagram illustrating a number of sensing zonesmonitored by the system of FIG. 1;

FIG. 4 is a block diagram of one representative embodiment of theaugmented reality display generator of FIG. 1;

FIG. 5 is a block diagram of another representative embodiment of anaugmented reality display generator in accordance with an aspect of thepresent disclosure;

FIG. 6 is a flow diagram illustrating one representative embodiment ofan augmented reality display method in accordance with an aspect of thepresent disclosure; and

FIGS. 7-10 illustrate embodiments of one or more components of thesafety system incorporated at various locations around the vehicle.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appendeddrawings, where like numerals reference like elements, is intended as adescription of various embodiments of the disclosed subject matter andis not intended to represent the only embodiments. Each embodimentdescribed in this disclosure is provided merely as an example orillustration and should not be construed as preferred or advantageousover other embodiments. The illustrative examples provided herein arenot intended to be exhaustive or to limit the claimed subject matter tothe precise forms disclosed.

The present disclosure relates to a system of information gatheringdevices, displays and associated programmed hardware, and their methodsof use, that provide, for example, increased driver visibility and blindspot prevention in vehicles, such as Class 8 trucks. The systems and/ormethods can be employed alone or can be employed to augment other blindspot prevention aids, such as side view mirrors, etc. In someembodiments, the system is configured to employ augmented realitytechniques and methodologies in order to “see-through” an obstruction inthe driver's view. For example, it is known that a driver's view in atractor-trailer combination is very limited when changing lanes orbacking up, for example, into a tight loading dock, due to the presenceof occluding vehicle objects, such as the semi-trailer, the sleeper cab,if equipped, other cab structure, combinations thereof, etc. As a resultof the occluding objects being “virtually” removed by the systems andmethods of the present disclosure, improved visibility to the driver isprovided, and increased safety and reduced property damage may beachieved.

As will be described in more detail below, digital cameras, radar,lidar, thermal imaging devices and/or similar information gatheringdevices can be placed at various locations around the vehicle and/orassociated trailer, if equipped. Additionally, one or more displays areplaced around the vehicle at locations that may correspond to actualview points of the vehicle driver. Some examples of the placement ofdisplays that will be described in more detail below include thefollowing: (1) a display provided at the rear of the sleeper or day cabin order to present the rear scene looking behind an associated trailer,as shown in FIG. 7; (2) a display provided with the rear view mirror, asshown in FIG. 8; (3) a display provided with the side view mirrors toaugment the viewing capabilities of the mirrors, as shown in FIG. 9; (4)one or more displays provided on each side of the driver seat, as shownin FIG. 10.

The term “augmented reality” as used throughout this detaileddescription and in the claims refers to any rendered image, article orobject using a combination of real-world views that are merged withcomputer generated images. For purposes of clarity, the terms “real” and“virtual” are used throughout this detailed description and in theclaims to distinguish between various types of images and/or objects.For example, a real view or real image refers to any view or image of areal environment that is occupied by a user. These views are typicallyreproduced with still or video cameras. In contrast, a virtual image orvirtual object is any image or object that is generated by a computingdevice and which is associated with a virtual environment. Moreover, forpurposes of clarity, the term “virtual design element” is usedthroughout this detailed description and in the claims to refercollectively to any type of virtual object, virtual image or virtualgraphic that may be created by, or used with, the system.

An augmented reality environment can be created by the combination ofvirtual images or objects with a real views or images. In someembodiments that will be described in more detail below, the realobjects or images are provided naturally by a mirror or like reflectivesurface or a transparent surface, such as a window. In otherembodiments, the real objects or images are generated by, for example,one or more cameras and/or the like. It will be appreciated that thegeneration of an augmented reality environment or scene can use a singlesource of information, or a combination of any two or more sources ofinformation described herein.

In the following description, numerous specific details are set forth inorder to provide a thorough understanding of one or more embodiments ofthe present disclosure. It will be apparent to one skilled in the art,however, that many embodiments of the present disclosure may bepracticed without some or all of the specific details. In someinstances, well-known process steps have not been described in detail inorder to not unnecessarily obscure various aspects of the presentdisclosure. Further, it will be appreciated that embodiments of thepresent disclosure may employ any combination of features describedherein.

Although representative embodiments of the present disclosure isdescribed with reference to Class 8 trucks, it will be appreciated thataspects of the present disclosure have wide application, and therefore,may be suitable for use with many types of vehicles, such as passengervehicles, buses, RVs, commercial vehicles, light and medium dutyvehicles, and the like. Accordingly, the following descriptions andillustrations herein should be considered illustrative in nature, andthus, not limiting the scope of the claimed subject matter.

Turning now to FIG. 1, there is shown a schematic diagram of one exampleof a vehicle safety system, generally designated 20, in accordance withaspects of the present disclosure. The system 20 may be installed in asuitable vehicle (sometimes referred to herein as the “host vehicle”)for providing one or more benefits to the driver, such as improveddriver visibility, reduction of blind spots, etc. This may includedetecting or sensing an environment composed of one or more foreignobjects (e.g. target object(s)) in relation to the host vehicle, which,for example, could pose a potential safety concern to the driver of thehost vehicle, to a pedestrian in the vicinity of the host vehicle, to adriver of an adjacent vehicle, etc. The system 20 is capable ofdetecting or sensing a wide variety of different target objects,including both moving and non-moving objects. For example, the targetobject can be a vehicle in an adjacent lane (e.g., a “side vehicle”) ora vehicle approaching the vehicle from behind (e.g., a “rear trailingvehicle”). The target object may also be a pedestrian or animal eitherstationarily positioned or crossing behind the host vehicle, etc., ormay be stationary, inanimate objects, such as trees, barriers,buildings, street signs, etc., on the periphery of or behind thevehicle.

As mentioned above, the target object(s) may be located in the blindspot or occluded area of the host vehicle. In that regard, FIG. 2illustrates various blind spots common to conventional vehicles, such asa tractor-trailer combination. As shown in FIG. 2, typical blind spotsinclude an area 40 located at the driver's side of the vehicle caused bythe A pillar, the B pillar, the sleeper section or other structure ofthe cab. Area 40 is typically not accessible by the driver side mirrors.The blind spots also include an area 42 located behind the trailer. Theblind spots also include an area 44 located at the passenger's side ofthe vehicle and at an angle with respect to the vehicle caused by the Apillar, the B pillar, the sleeper section or other structure of the cab.Area 44 is typically not accessible by the passenger side mirrors. Insome instances, portions of area 44 may be slightly accessible by theside mirrors. The blind spots may also include an area 46 in front ofthe vehicle and to the passenger side of the vehicle caused by the frontsection/hood of the vehicle. Area 46 also extends rearwardly to includethe area on the passenger side adjacent the vehicle front section/hood.

To sense one or more target objects in the vicinity of the vehicle, thevehicle safety system 20 collects information from various informationsources 24 associated with the host vehicle. In some embodiments, thecollected information represents data associated with the vehiclesurroundings, sometimes referred to as the vehicle environment. In oneembodiment, the collected information represents data associated atleast in part with one or more blind spots of the vehicle driver,including areas 40, 42, 44, and 46. The information sources 24 caninclude, for example, devices such as digital cameras, radar, lidar,thermal imaging cameras, etc., which are mounted on or otherwiseassociated with the host vehicle in suitable locations for obtaininginformation related to the driver's various blind spots or otheroccluded areas. In another embodiment, the information sources 24 mayinclude devices discrete from vehicle, such as traffic cameras, roadsidebeacons, components of system 20 or a similar system installed onthird-party vehicles, which communicate with the host vehicle viacellular, short or long range RF, or similar protocols, and provideinformation related to the driver's various blind spots or otheroccluded areas. In these or other embodiments, the information sources24 may also optionally include devices that collect or generate dataindicative of vehicle operating parameters, such as vehicle speed,vehicle acceleration, etc.

In response to information collected by one or more of these informationsources 24, or any combination thereof, the system 20 presents to thedriver with the aid of one or more displays an augmented realityenvironment comprising a real image depicting a scene from the viewpointof the driver and virtual design elements (e.g., person, animal,barrier, road, terrain, etc.) that are located in one of the driver'sblind spots or occluded areas. In some embodiments, the virtual designelements also include the object (e.g., trailer, vehicle structure(e.g., hood, cab, etc.), etc.) that is occluding the view of the driver.As a result, the presence of the virtual design elements allows thedriver to “see through” the occluding structures, such as the trailer,in order to increase driver visibility, etc.

Still referring to FIG. 1, the components of the system 20 will now bedescribed in more detail. As shown in the embodiment of FIG. 1, thesystem 20 includes one or more information sources 24, an augmentedreality display generator 28, and one or more displays 32. The displaygenerator 28 is either directly connected in communication with one ormore information sources 24 or can be connected to the one or moreinformation sources 24 via a vehicle wide network 36, such as acontroller area network (CAN). Those skilled in the art and others willrecognize that the vehicle-wide network 36 may be implemented using anynumber of different communication protocols such as, but not limited to,Society of Automotive Engineers' (“SAE”) J1587, SAE J1922, SAE J1939,SAE J1708, and combinations thereof. Direct connection can be carriedout either wired or wirelessly, or both.

The information sources 24 in some embodiments can include but are notlimited to digital cameras or other image gathering devices, opticalsensors, radar, lidar, ultrasonic or other RF sensors, thermal imagingcameras, thermal sensors, proximity sensors, etc. In use, for example, asingle device or sensor or a combination of two or more of these devicesand/or sensors is capable of generating vehicle environment data, whichmay, for example, contain camera images, an infrared image, etc., of theenvironment surrounding the host vehicle. As will be described in moredetail below, the information contained in this vehicle environment datacan be used by the system 20 to either generate real images, virtualimages, or both.

In some embodiments, the information generating sources 24 are mountedto or otherwise associated with the host vehicle at one or more desiredinformation gathering locations. As can be appreciated, the location andnumber of devices that are used will depend upon the particularapplication and can be readily modified as conditions dictate. In theembodiment shown in FIG. 3, the information sources 24 are placed aroundhost vehicle (shown as a tractor trailer combination) so as to form sidesensing zones 50 and 52 and a rear sensing zone 54. In one embodiment,one or more information sources 24 can also be located around the hostvehicle so as to form a front sensing zone 56. In another embodiment,one or more information sources 24 can also be located at the rear ofthe lead vehicle (e.g., tractor) so as to form a gap sensing zone 58.

In some embodiments, additional information sources 24 can be optionallyemployed in order to carry out one or more functions of the system 20.In that regard, some embodiments of the system 20 also employ variousvehicle system sensors or the like, including brake sensors, wheel speedsensors, a vehicle speed sensor, transmission gear sensor,accelerometers, a steering angle sensor, etc. Information from theseadditional information sources can be used in conjunction with theinformation sources associated with the sensing zones 50, 52, 54, 56,and 58 in some embodiments in order to carry out various functionalityof the system 20.

At least one of the information sources 24 of the vehicle safety system20 in some embodiments may optionally include a data acquisition unitthat comprises one or more receivers. In these embodiments, the dataacquisition unit is configured to receive, for example, information frominformation sources discrete from the host vehicle, such as short-rangecommunication devices (transmitters or the like from other vehicles inthe vicinity of the host vehicle that are equipped with the system 20 orsimilar functionality, road side or traffic intersection beacons,traffic cameras, etc.). Information that can be transmitted to thevehicle 20 includes but is not limited to one or more of the following:vehicle operating data, blind spot data related to the host vehicle orto the transmitting vehicle, and incident data. In some embodiments, thedata acquisition unit may also include transmitters or can be equippedwith transceivers in order to transmit information generated by system20 to other vehicles in the vicinity of the host vehicle.

In one embodiment, the system 20 may be used in conjunction with othervehicle safety systems or functionality, such as adaptive cruisecontrol, autonomous driving, collision avoidance, collision warning,lane departure warning, lane change/merge detection, object detection,vehicle path prediction, rear impact collision warning/avoidance, roadcondition detection, just to name a few. In that regard, the system 20in one embodiment is configured to receive and/or share data with theseoptional vehicle systems in order to carry out the functionality of thesystem 20.

The information from at least one these information sources 24, or anycombination of these information sources 24, can be processed by thedisplay generator 28 or other components so that an augmented realityenvironment can be presented to the vehicle driver with the aid of oneor more of the displays 32. As was described briefly above and will bedescribed in more detail below, the augmented reality environment insome embodiments is created by the combination of a real image and oneor more virtual design elements, which is presented together to thevehicle driver.

In various embodiments of the system 20, the one or more displays 32 caninclude a generally opaque display, for example, a liquid crystaldisplay (LCD), a light emitting polymer display (LPD), a plasma display,or a light emitting diode (LED) display. In these embodiments, theaugmented reality environment can be presented to the driver entirely bythe opaque display. In other embodiments of the system 20, the one ormore displays can include transparent displays or “see through”displays, such as transparent LCD, OLED or Head-up displays (HUD). Inone embodiment, the transparent display can be fabricated as a layer ofOLEDs sandwiched between two transparent pieces of film (e.g., siliconor plastic film, etc.). In these embodiments, as will be described inmore detail below, the transparent displays can be either mounteddirectly over a mirror of the vehicle, such as a rearview mirror, a sideview mirror, etc., or can overlay a vehicle window or sections thereof,such as a rear window or front windshield of the vehicle. As such, theaugmented reality environment is presented to the vehicle driver by acombination of a reflective or transparent layer (e.g., mirror, window,etc.) of the vehicle, which allows real images to be presented naturallyto the driver via transmission of light, and a transparent display 32,which provides the virtual design elements to the driver.

In accordance with an aspect of the present disclosure, the displaygenerator 28 is configured to: (1) collect information from one or moreinformation sources 24; (2) generate virtual design elements based onthe collected information; and (3) present the augmented realityenvironment or portions thereof to the vehicle driver via at least oneof the one or more displays 32. As will be described in more detailbelow, the virtual design elements can include target objects, such aspeople, animals, posts, building structure, etc., as well as portions ofthe environment occluded by the host vehicle. As presented to thevehicle driver, the augmented reality environment provides a “seethrough” effect in order to represent information to the driver thatwould be normally hidden or obscured from view.

Turning now to FIG. 4, there is shown in block diagrammatic form onerepresentative embodiment of the display generator 28 formed inaccordance with an aspect of the present disclosure and capable ofcarrying out the functionality described above. As shown in FIG. 4, thedisplay generator 28 includes one or more modules. In the embodimentshown, the display generator 28 includes an information gathering module62, an augmented reality rendering module 66, and a display module 72.While the modules are separately illustrated in the embodiment shown, itwill be appreciated that the functionality carried out by each modulecan be combined into fewer modules or further separated into additionalmodules. In some embodiments, the modules of the display generator 28contain logic rules for carrying out the functionality of the system.The logic rules in these and other embodiments can be implemented inhardware, in software, or combinations of hardware and software.

Still referring to FIG. 4, the information gathering module 62implements logic for obtaining real-time or near real time data from theinformation sources 24. The data can include images, video, etc.,associated with one or more of the side sensing zones 50 and 52, therear sensing zone 54, the front sensing zone 56, and the gap sensingzone 58. In some embodiments, only one zone is needed to generate theaugmented reality environment. In other embodiments, a combination oftwo or more zones is used to generate the augmented reality environmentor scene. The data can also optionally include vehicle operating data,or data from external sources (third party vehicles, beacons, trafficcameras, etc.) representing images or video associated with one or moreof the various sensing zones. During the acquisition process, datareceived from the information sources 24 can be processed and temporarystored, such as in memory and/or an associated buffer.

The augmented reality rendering module 66 implements logic forgenerating virtual design elements for the augmented reality environmentbased on information obtained from the information gathering module 62.In doing so, the augmented reality rendering module 66 can interpretvarious types of information and employ various augmented realityrendering engines for generating the augmented reality environment. Inone embodiment, the module 62 can convert radar, lidar, and/or thermalimaging into virtual design elements that graphically represent a scene,an image, or objects therein that are hidden or occluded from view ofthe driver. In another embodiment, the module 66 converts a camera imageinto virtual design elements that graphically represent a scene, animage, or objects therein that are hidden or occluded from view of thedriver. In some embodiments, the augmented reality rendering module 78also implements logic for presenting real images for the augmentedreality environment based on information obtained from the informationgathering module 62. In some of these embodiments, the module 62combines the real images and the virtual images in a suitable manner toform the augmented reality environment.

As further illustrated in FIG. 4, the display generator 28 furtherincludes a display module 72. The display module 72 implements logic forcausing the virtual design elements generated by the augmented realityrendering module 78 to be presented to the display 32 for display. Insome embodiments, the display module 72 is further configured to presentthe virtual design elements together with the real images for display.It will be appreciated that know image processing, buffering, and/or thelike can occur at one or more of the modules 62, 66, and 72.

FIG. 5 illustrates another suitable embodiment of the display generator28 in block diagrammatic form. As shown in FIG. 5, the display generator28 includes a processor 76 and memory 78. The memory 78 may includecomputer readable storage media in read-only memory (ROM), random-accessmemory (RAM), and keep-alive memory (KAM), for example. The KAM may beused to store various operating variables or program instructions whilethe processor 76 is powered down. The computer-readable storage mediamay be implemented using any of a number of known memory devices such asPROMs (programmable read-only memory), EPROMs (electrically PROM),EEPROMs (electrically erasable PROM), flash memory, or any otherelectric, magnetic, optical, or combination memory devices capable ofstoring data, instructions, programs, modules, etc. In the embodimentshown, a data acquisition module 62, an augmented reality 66, and adisplay module 66 are stored in memory 78. In some embodiments, thedisplay generator 28 may include additional components including but notlimited to, analog to digital (A/D) and digital to analog (D/A)circuitry, input/output circuitry and devices (I/O) and appropriatesignal conditioning and buffer circuitry.

As used herein, the term processor is not limited to integrated circuitsreferred to in the art as a computer, but broadly refers to amicrocontroller, a microcomputer, a microprocessor, a programmable logiccontroller, an application specific integrated circuit, otherprogrammable circuits, combinations of the above, among others.Therefore, as used herein, the term “processor” can be used to generallydescribe these aforementioned components, and can be either hardware orsoftware, or combinations thereof, that implement logic for carrying outvarious aspects of the present disclosure. Similarly, the term “module”can include logic that may be implemented in either hardware orsoftware, or combinations thereof.

FIG. 6 is a flow diagram that depicts one exemplary embodiment of anaugmented reality display method 600 formed in accordance with thedisclosed subject matter. In one embodiment, the method 600 may beimplemented by the modules 62, 66, and 72 of the display generator 36from either FIG. 4 or 5. Accordingly, information may be collected orotherwise received from one or more information sources 24, convertedinto an augmented reality environment or virtual design elementsthereof, and presented to the vehicle driver with the aid of one or moredisplays 32. As a preliminary matter, those skilled in the art willappreciate that such functionality is typically designed to be carriedout in a continual manner, i.e., once initialized and operating, thedisplay generator 28 continually monitors and displays information.Accordingly, the method 600 operates continually until the displaygenerator is powered down or its operation is otherwise interrupted.

As illustrated in FIG. 6, the routine 600 begins at block 602 where astart-up event occurs that will cause an augmented reality environmentto be presented to the vehicle driver with the aid of one or moredisplays 32. Generally described, a start-up event is an event type thatwill cause the display 32 to transition from an inactive state to anactive state. By way of example only, the start-up event that occurs atblock 602 may be the ignition of the vehicle's engine, which results inpower being supplied to an ignition bus. Also, the display 32 may be putto “sleep” in a reduced power state when the vehicle is inactive for apredetermined period of time. Thus, the start-up event may be anothertype of event, such as the return of the display 32 from a reduced powerstate.

If a start event occurs at block 602, the method 600 proceeds to block604, where the display generator 28 begins collecting information fromthe one or more information sources 24 indicative of one or more eventsoccurring in one or more of the sensing zones 50, 52, 54, 56, and 58.Next, at block 606, the display generator 28 renders one or more virtualdesign elements 80 representative of occluded objects located in one ormore of the sensing zones for subsequent display. In one embodiment, thevirtual design elements 80 are rendered based on the driver's viewpoint. In some embodiments, the virtual design elements 80 can include,for example, a general outline of the occluding structure, such as thetrailer, and any target objects that may be occluded thereby. Forexample, the virtual design elements 80 can include an animal 80B, theroad 80C, and the terrain 80D, that is normally occluded by the trailer,as shown in FIG. 7. The virtual design elements 80 can also includevehicle structure 80A, such as the outline of the trailer, that isresponsible for the occluded view. In some embodiments, the renderedvirtual design elements 80 can be temporarily stored in memory 58 or anassociated buffer. This information may be continually collected andprocessed so that current events can be conveyed on one or more displays32.

From block 606, the method proceeds to block 608, where the virtualdesign elements are then presented to a display 32 for display. Oncereceived by the display 32, the virtual design elements are rendered bydisplay 32, as shown in the examples of FIGS. 7-10. In some embodimentsthat employ an opaque display, the virtual design elements 80 arepresented to the display 32 in conjunction with real images 82. Forexample, real images 82 can be obtained or converted from theinformation provided from the information sources 24. In thisembodiment, the display generator 28 overlays, superimposes or otherwisecombines the virtual design elements 80 with the real images 82 to forman augmented reality environment at block 610 for display. In otherembodiments, the display generator 28 takes the real image of thevehicle environment and converts only that portion of the real imagethat is occluded from the view of the driver into virtual designelements 80 in order to form an augmented reality environment. In someembodiments, the real images 82 can be temporarily stored in memory 58or an associated buffer. This information may be continually collectedand processed so that current events can be conveyed on one or moredisplays 32.

The method 600 then proceeds to block 612, where a determination is madeas to whether a process termination event has occurred. The terminationevent can be turning the ignition key to the “off” position, poweringdown the system 20 or one or more displays 32, or placing one or more ofthe displays 32 in sleep or stand-by mode, etc. If a termination eventoccurs at block 612, then the method 600 ends. If not, the methodreturns to block 604 so that a continuous feed is presented to thedisplay 26.

It should be well understood that the routine 600 described above withreference to FIG. 6 does not show all of the functions performed whenpresenting the augmented reality environment to the driver. Instead, theroutine 600 describes exemplary embodiments of the disclosed subjectmatter. Those skilled in the art and others will recognize that somefunctions may be performed in a different order, omitted/added, orotherwise varied without departing from the scope of the claimed subjectmatter.

Carrying out the one or more embodiments of the method 600 results inaugmented reality environments depicted schematically in the examples ofFIGS. 7-10. For ease of illustration, the real images 82 are shown withthicker lines, which appear darker in the FIGURES, while the virtualdesign elements are shown with thinner lines, which appear lighter inthe FIGURES.

FIG. 7 is a schematic representation of a display 32 employed in lieu ofthe rear window of the host vehicle. As shown in FIG. 7, the augmentedreality environment is created by the virtual design elements 80A-80Dand the real image 82 presented by the display 32. In the embodimentshown, the virtual design elements include the outline of the occludingstructure 80A, the animal 80B, the road 80C, and the terrain 80D, whichare normally occluded by the trailer. In generating the virtual designelements 80, information based on sensing zones 50, 52, 54, and/or 58can be used. In this embodiment, the real image 82 includes the scenebehind the tractor that is not occluded by the trailer. In otherembodiments, instead of an opaque display, a transparent display can beused in conjunction with the rear window in order to present theaugmented reality environment to the driver.

FIG. 8 is a schematic representation of a display 32 mounted over asection of the rear view mirror 90. As shown in FIG. 8, the augmentedreality environment is created by the virtual design elements 80presented by display 32 and the real image 82 presented by thereflective surface of the mirror 90. In the embodiment shown, thevirtual design elements include the trailer outline 80A, the road 80C,and the terrain 80D normally occluded by the trailer.

FIG. 9 is a schematic representation of a display 32 mounted over asection of the side view mirror 96. As shown in FIG. 9, the augmentedreality environment is created by the virtual design elements 80presented by display 32 and the real image 82 presented by thereflective surface 98 of the mirror 96. Of course, some embodiments canemploy an opaque display in lieu of the side mirror, as well. In theembodiment shown, the virtual design elements 80 include building 80E,portions of the flag 80F, and the outline of the trailer 80A,

FIG. 10 is a schematic representation of two displays 32A and 32Bmounted on the sides of the driver seat 100. As shown in FIG. 10, theaugmented reality environment is created by both the virtual designelements 80 and the real images 82 presented by displays 32A and 32B. Inthis embodiment, the augment reality environment includes the occludedareas on the passenger and driver side of the trailer, which may bebased on information from sensing zones 50, 52, and/or 54. The augmentreality environment presented by such displays 32A and 32B can be usedin conjunction with either traditional side mirrors, or side mirrorsconfigured as described in FIG. 9. Of course, some of the informationregarding the side sensing zones can be obtained from other vehicles inthe vicinity of the host vehicle.

Other applications of one or more embodiments of the system 20 arecontemplated in accordance with one or more aspects of the presentdisclosure. For example, the system 20 in one embodiment may bealternatively or additionally configured to employ a heads up display(HUD) as one of the displays 32 for presenting different configurationsof the hood to the driver. In one embodiment, the display generator 28of the system 20 is configured to generate virtual design elements inthe form of the vehicle hood and objects that are present in area 46(See FIG. 2) but occluded by the front section/hood of the vehicle. Inanother embodiment, the display generator 28 of the system 20 isconfigured to generate either virtual design elements or a realrepresentation of a vehicle hood from a different vehicle model. Forexample, the host vehicle may be a commercial truck, such as thePeterbilt® 389 semi truck that is equipped with a “long” hood, but thesystem 20 may present through the HUD a virtual hood representative of ashorter version of the hood, sometimes referred to as the “aero” hood,or vice versa.

In yet other embodiments, the system 20 may utilize information fromother systems 20 installed in either trailing vehicles or leadingvehicles. For example, if the host vehicle is part of a vehicle platoon(i.e., two or more vehicles one in front of the other), the system 20 ofthe host vehicle can communicate with the other vehicle(s) in order toprovide the driver of the host vehicle with an augmented realityenvironment of what is in front of the lead vehicle, or what is behindthe trailing vehicle. As such, the augmented reality environmentpresented by the system 20 of the host vehicle allows the driver to “seethrough” the lead vehicle (the lead vehicle transmits information fromin front of the lead vehicle, including area 46), thereby reducingdriver eye fatigue, or allows the driver to “see through” the trailingvehicle (the trailing vehicle transmits information from behind thetrailing vehicle, including from area 42), thereby providing the driverwith additional information regarding the environment.

The principles, representative embodiments, and modes of operation ofthe present disclosure have been described in the foregoing description.However, aspects of the present disclosure which are intended to beprotected are not to be construed as limited to the particularembodiments disclosed. Further, the embodiments described herein are tobe regarded as illustrative rather than restrictive. It will beappreciated that variations and changes may be made by others, andequivalents employed, without departing from the spirit of the presentdisclosure. Accordingly, it is expressly intended that all suchvariations, changes, and equivalents fall within the spirit and scope ofthe present disclosure, as claimed.

The embodiments of the invention in which an exclusive property orprivilege is claimed are defined as follows:
 1. A computer-implementedmethod of providing information to a driver of a host vehicle in avehicle platoon, the method comprising: receiving, by a displaygenerator of the host vehicle, video data from a trailing vehicle,wherein the trailing vehicle is separate from and behind the hostvehicle, and wherein the video data depicts an area behind the trailingvehicle that is occluded from view of the driver of the host vehicle bythe trailing vehicle; and presenting, by the display generator, thevideo data from the trailing vehicle to the driver of the host vehicle.2. The method of claim 1, further comprising: causing one or moredisplays of the host vehicle to render one or more virtual designelements representative of one or more objects located in the sceneoccluded from view of the first driver.
 3. The method of claim 1,wherein presenting the video data from the trailing vehicle to thedriver of the host vehicle includes: generating one or more virtualdesign elements representative of one or more objects located in thearea occluded from view of the first driver; obtaining a real image ofthe area, the real image based on the point of view of the first driver;and causing the one or more virtual design elements and the real imageto be displayed together by one or more displays of the host vehicle. 4.The method of claim 3, wherein causing the virtual design elements andthe real image to be displayed together by the one or more displays ofthe host vehicle includes: arranging the one or more virtual designelements over a preselected area of the real image.
 5. The method ofclaim 1, further comprising: obtaining, by the display generator of thehost vehicle, additional data from a source discrete from the hostvehicle, the source discrete from the host vehicle including one of abeacon and a traffic camera.
 6. The method of claim 1, whereinpresenting the video data from the trailing vehicle to the driver of thehost vehicle includes presenting the video data on at least one displaymounted to a side of a driver seat of the host vehicle.
 7. The method ofclaim 1, wherein presenting the video data from the trailing vehicle tothe driver of the host vehicle includes presenting the video data on atleast one display mounted in lieu of a rear window of the host vehicle.8. A non-transitory computer-readable medium having modules storedthereon that, in response to execution, cause a host vehicle in avehicle platoon to convey information to a driver of the host vehicle,the modules comprising: an information gathering module configured toreceive video data from a trailing vehicle, wherein the trailing vehicleis separate from and behind the host vehicle, and wherein the video datadepicts an area behind the trailing vehicle that is occluded from viewof the driver of the host vehicle by the trailing vehicle; and a displaymodule configured to cause the video data from the trailing vehicle tobe presented to the driver of the host vehicle via a display.
 9. Thecomputer-readable medium of claim 8, wherein the modules furthercomprise an augmented reality image rendering module configured togenerate one or more virtual design elements representative of objectsoccluded from view of the driver of the host vehicle based oninformation received from one or more information sources; and whereinthe display module is further configured to cause the video data to bepresented with the virtual design elements, wherein the video data andthe virtual design elements together form an augmented realityenvironment.
 10. The computer-readable medium of claim 9, wherein theaugmented reality image rendering module is further configured toarrange and align the generated one or more virtual design elements overa preselected area of the video data.
 11. The computer-readable mediumof claim 9, wherein the one or more information sources includeinformation sources selected from the group consisting of a digitalcamera, a thermal imaging device, radar, and lidar.
 12. Thecomputer-readable medium of claim 8, wherein causing the video data fromthe trailing vehicle to be presented to the driver of the host vehiclevia a display includes causing the video data from the trailing vehicleto be presented to the driver of the host vehicle via a display mountedto a side of a driver seat of the host vehicle.
 13. Thecomputer-readable medium of claim 8, wherein causing the video data fromthe trailing vehicle to be presented to the driver of the host vehiclevia a display includes causing the video from the trailing vehicle to bepresented to the driver of the host vehicle via a display mounted inlieu of a rear window of the host vehicle.
 14. A system for providinginformation to a driver of a host vehicle in a vehicle platoon, thesystem comprising: one or more displays; and a display generatorcommunicatively coupled to a trailing vehicle, wherein the trailingvehicle is separate from and behind the host vehicle; wherein thedisplay generator is configured to: receive video data from the trailingvehicle, wherein the video data depicts an area behind the trailingvehicle that is occluded from view of the driver of the host vehicle bythe trailing vehicle; and present the video data from the trailingvehicle to the driver of the host vehicle using the one or moredisplays.
 15. The system of claim 14, wherein the one or more displaysinclude a transparent display configured to overlay one of a transparentlayer and a reflective layer.
 16. The system of claim 14, wherein thetransparent layer includes a window and the reflective layer includes amirror.
 17. The system of claim 16, wherein the mirror includes one of aside mirror and a rear view mirror.
 18. The system of claim 14, whereinthe display generator includes a memory storing one or more moduleshaving program instructions for conveying information on the one or moredisplays; and a processor configured to execute the program instructionsof the one or more modules.
 19. The system of claim 14, wherein thedisplay generator is further communicatively coupled to one or moreinformation sources that include information sources selected from thegroup consisting of a digital camera, a thermal imaging device, radar,lidar, a beacon, and a traffic camera.
 20. The system of claim 14,wherein the one or more displays include at least one display mounted toa side of a driver seat of the host vehicle or a at least one displaymounted in lieu of a rear view window of the host vehicle.